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ABSTRACT 

GUIDON is an intelligent computer-aided instruction 
(ICAI) program for teaching diagnosis, which has been tested using 
the infectious disease diagnosis rules of the MYCIN consultation 
system developed at the Stanford University School of Medicine. 
5UXD01Lengages_a_ student in a dialogue about a patient suspected of 
having an infection and thus teaches the student about relevant 
clinical and laboratory data and diagnosis of the causative organisms 
Without reprogramming, the program can discuss any diagnostic problem 
that it can solve on its own. Moreover, by substituting problem 
solving knowledge from other domains, the program can immediately 
discuss problems in those domains. This power derives from the use of 
artificial intelligence methods for representing independently both 
subject material and general knowledge about how to teach. There are 
-teaching rules and procedures for: determining what the student 
knows, responding to his/her partial solution, providing hints, and 
opportunistically interrupting to test his/her understanding. 
Experience with GUIDON reveals the importance of differentiating 
between causal and strategic knowledge in order to explain diagnostic 
rules and to teach a reasoning approach. These lessons are now 
guiding the development of new representations for teaching. A 
13-item bibliography and a list of names and addresses of government 
and private sector research/information centers and personnel 
concerned— with computer-aided instruction are provided. 
( Author /ESR) 
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William J. Clancey 
Stanford University 

GUIDON is an intelligent computer-aided instructional (ICAI) program for teaching diagnosis, such as 
medical diagnosis, The program is general. Without reprogramming, the program can discuss with a 
student any diagnostic problem that it can solve on its own. Moreover, by substituting problem -solving 
knowledge from ^ her domains, the program can immediately discuss problems in those domains. This 
power derives freer, the use of Artificial Intelligence methods for representing both subject material and 
knowledge about how to teach. These are represented independently, so the teaching knowledge is 
general. Tnere are teaching rules and procedures for: determining what the student knows, responding to 
his nartial solution, providing hints, and opportunistically interrupting to test his understanding. Ex* 
perience with GUIDON reveals the importance of separating out casual and strategic knowledge in order 
to explat.i diagnostic rules and to teach a reasoning approach, These lessons are now guiding the 
development of ne v representations for teaching. 



GUIDON, a program for teaching diagnostic 
problem-solving, is being developed by William J. 
Clancey and his colleagues at Stanford University. 
Using the rules of the MYCIN consultation system 
(Shortliffe, 1976) as subject material, GUIDON 
engages a student in a dialogue about a patient 
suspected of having an infection. In this manner, it 
teaches the student about the relevant clinical and 
laboratory data and about how to use that information 
for diagnosing the causative organism. GUIDON's 
mixed-initiative dialogue differs from that of other 
ICAI programs in its use of prolonged, structured 
teaching interactions that go beyond responding to the 
student's last move [as in WEST (Burton and Brown, 
1979) and WUMPUS (Goldstein, 1979)] and repetitive 
questioning and answering (as in SCHOLAR (Car- 
bone 11, 1970) and WHY (Stevens, Collins, and Goldin, 
1982)1. 

MYCIN'S infectious-disease diagnosis rules con- 
stitute the skills to be taught. As applied to a particular 
problem, the rules provide GUIDON with topics to be 
discussed and with a basis for evaluating the student's 
behavior. GUIDON's teaching knowledge is wholly 
separate from MYCIN. It is stated explicitly in the 
form of 200 tutorial rules, which include methods for 
guiding the dialogue economically, presenting 
diagnostic rules, constructing a student model, and 
responding to the student's initiative. Because of the 
separation of teaching and dpmain knowledge, 
MYCIN'S infectious-disease knowledge base can be 

Revised article reprinted wjth permission from The Handbook of Artificial 
Intelligence. Vol. II * A. Barr and E.A. Feigenbaum (Eds.). William Kauf* 
mann. Inc.. 1982. 

' Reprint requests should be addressed to William J. Clancey, Department of 
Computer Science, Stanford University. Stanford, CA 94305. 

0 • .... • - •' 
ERIC 



replaced by diagnostic rules for another problem 
domain. 

The large and complex MYCIN knowledge base 
provides a unique opportunity to apply and extend 
ICAI technology for student modeling and mixed- 
initative dialogue. GUIDON is designed to explore two 
basic questions: First, how do the problem-solving 
rules, which perform so well in the MYCIN con- 
sultation system, measure up to the needs of a tutorial 
interaction with a student? Second, what knowledge 
about teaching might be added to MYCIN to make it 
into an effective tutorial program? MYCIN'S rules 
have not been modified for the tutoring application, 
but they are used in new ways, for example, for 
making up quizzes, guiding the dialogue, summarizing 
evidence, and modeling the student's understanding. 

Several design guidelines for the rules make it 
plausible that the rules would be a ggbd vehicle for 
teaching. First, they are designed to capture a 
significant part of the knowledge necessary for good 
problem solving. Formal evaluation of MYCIN 
demonstrated that its competence in selecting an- 
timicrobial therapy for meningitis and for bacteremia 
is comparable to that of the members of the infectious-^ 
disease faculty at the Stanford University School of 7 
Medicine, (where MYCIN was developed; see Yu et al., 
1979). Second, flexible use of the rule set is made 
possible by the provision of representational 
metaknowledge, which allows a program to take apart 
rules and to reason about the components (this 
knowledge describes the number and type of • 
arguments of primitive functions in the rule language). 
Finally, MYCIN'S rules, in contrast with Bayesian 
programs, are couched in terms familiar to human 
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experts, so it seems likely that reading back MYCIN'S 
line of reasoning to a student might be helpful to him 
(or her). 

After a brief overview of MYCIN, this article 
discusses the following aspects of a GUIDON tutorial 
dialogue: 

1 . The nature of the interaction 

2. The components of the student model 

3. The organization of teaching knowledge into 
discourse procedures 

4. The use of the student model 

5. Opportunistic tutoring 

6. Pedagogical principles behind the tutoring rules. 

The capability of GUIDON to tutor from a library of 
cases and for domains outside of medicine is also 
discussed. The final section outlines the lessons 
• learned about knowledge representation that are being 
lapplied to reconfigure the MYCIN rule base for its use 
in teaching. 

\ 

I ■ v 

\ 

Overview of M YCIN 

\ MYCIN is a program that was developed by a team 
of physicians and AI specialists. The program was 
designed to advise nonexperts in the selection of 
antibiotic therapy for infectious diseases. The 
knowledge base consists of approximately 450 rules 
that deal with diagnosis of bacteremia, meningitis, and 
cystitis infections. The rules are applied by backward 
chaining, working from high-order goals, such as 
"Determine whether the patient requires treatment," 
down to more specific subgoals, such' as "Determine 
whether the patient has high risk for tuberculosis." 
These goals and subgoals become the "topics" of a 
dialogue with GUIDON. A typical rule is, roughly 
stated, "If the patient has been receiving steroids, then 
his risk- for tuberculosis meningitis is increased." The 
rules[are modified by a certainty factor (CF), in- 
dicating the rule author's degree of belief, on a scale 
from I- 1 to 1 that the conclusion holds when the 
premise is know to be true. (In the GUIDON excerpts 
showii below, the CFs are shown in parentheses, e.g., 
"(.95)1") In a MYCIN consultation, the rules are 
chained together, working downward from the high- 
order gpals. The pirogram asks a queston when it needs 
more case data to apply a rule. Thus, a tree of goals 
and rules is constructed; The goals are OR nodes (any 
of a number of rules may help determine a goal) and 
the rules are AND nodes (all of the subgoals 
referenced in the premise must be known for the rule 
to apply)/ We call this AND/OR tree with rule 
evaluations and final conclusions about goals the 
solution of th e diagnostic problem. 



' INTERACTION WITH GUIDON 

An essential part of tutorial dialogue management is 
allowing the student to express himself. In a mixed- 
initiative program, provision must be made, for every 
kind of initiative on the part of the student. This in- 
cludes referri/.* back to an earlier topic to provide 
more details, changing the topic, requesting case data, 
posing a hypothesis, asking for help, and so on. We 
might summarize this by saying that we must allow the 
student to specify what he knows, what he wants to 
know more about, and what he wants to ignore. The 
annotated protocol excerpted below illustrates 
GUIDON'S flexibility in responding to the student's 
initiative. To this point in the dialogue, the student has 
been given basic data about the patient. He has already 
determined that the patient has an infection, has 
evidence that it might be meningitis, and is trying to 
determine what is causing the meningitis ("(he type of 
the infection"). Student input (indicated here by •*) is 
in the form of commands from a menu, discussed 
further below. 

The student asks for the data and subgoa ls relevant to 
the topic being discussed. These are extracted from 
MYCIN'S rules. 

••Factors 

The following factors will be useful to determine the type of 

the infection: 

3a. whether the meningitis is partially treated 
3b. whether a lumbar puncture has been performed on 
Pt538 

3c. the duration of the neurological signs 
3d. the CSF protein 
(•) 3e. whether Pt538 has a rash consistent with Herpes 
Zoster 

GUIDON keeps track of what has been discussed, 
MYCIN'S use of data, and conjectures about what the 
student knows. 

Factors marked with (•) have already been stated. Based on 
what we have already mentioned, you should be able to 
conclude about: 3b. 

The student asks how a particular datum is useful. He is 

given the case-specific information and then a summary 
* of its use in the rule or rules that apply in this case 

(negative conclusions are omitted here because the 

choices are mutually exclusive). 

••Use 3c \ 

The duration of the neurological signs is 7.2 hours. 
Using other information available to you now, the duration 
of the neurological signs enables you to conclude that the 
type of the infection is bacterial (.2) viral (.2).[RULE524J. 

The student indicates that he has determined a par- 
ticularsubgoal. As described in this article, a ''discourse 
procedure" is invoked to respond to the student. If the 
student model were not consistent with his claim, he' 
would be asked to state his conclusion and then possibly 
to support it. - 
••IKNOW3|> 

Good. MYCIN has also made a decision. 
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The site of CULTURE- 1 is evidence that a lumbar puncture 
has been performed on Pt538 [RULEII2J. 

Back to our discussion of the type of the infection... 

At this point, the topic of discussion is completed and 
the student is receiving a summary of the relevant 
conclusions. The program is automatically extracting 
the "key factor" from each rule and displaying only the 
4 'interesting * * (useful/ conclusions. 

Summary of evidence that the type of the infection is bac- 
terial (.98): 

29a. The fact that the meningitis is partially treated (.95) is 
evidence that the type of the infection is bacterial (.19) 
, [RULEI48] 

29b. The WBC from the CSF and the percent of PMNs in the 
CSF is evidence that the type of the infection is bacterial 
(.9) [RULESOI] 

The i'tudent requests more information about one of the 
inferences. The program reasons that because several 
subgoals ha ve not been discussed, these are the details to 
offer first. 

••Detail* 29a 

For which of the following factors would you like to see a 
, review of the evidence? 
(Select from the following): 

1 . the infection which requires therapy 

2. the fact that the meningitis is partially treated (.95) 

1 . " »i 

Summary of evidence that the meningitis is partially treated 
(.95): 

32a. The infection which requires therapy, the fact that 
organisms were not seen on the stain of the pending CSF 
culture, and the time since therapy with the cephaloihin 
was started are evidence that the meningitis is partially 
treated (.95) [RULEI45] 

Given this information, perhaps the student would like 
to see the details of how it is used. 

Do you want to see RULE 148? 

••No 

GUIDON provides a menu of options for requesting 
case data, asking for MYCIN'S evaluation of the 
problem (e.g., "What subgoals are PENDING?" 
"Give me DETAILS"), determining dialogue context 
(e.g., "What RULE are we discussing?"), changing 
the topic, requesting assistance (the options HELP, 
HINT, and TELLME), and conveying what is known 
(e.g., "I want to make a HYPOTHESIS"). The menu 
of over 30 options allows for input to be terse, while 
defining clearly for jhe student what the program can 
understand. As arguments to the options, the student 
can use phrases (e.g., "IKNOW about the lumbar 

. puncture"), keywords (e.g., "IKNOW LP"), or 
indices of remarks made by the program (e.g., 

fr "IKNOW 3B"). All of the output text is generated 
-from short phrases ('-the following factors," "the 
CSF protein," "is evidence that") with verb tense and 
number adjusted according to context. GUIDON'S 
initiatives involve probing the student's understanding 
(if a question or hypothesis is unexpected), offering , 
overviews and summaries, introducing new topics 7 
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when rules are being discussed and suggesting that a 
topic be terminated. These capabilities are discussed in 
the sections below on alternative dialogues, respon- 
ding to partial solutions, and opportunistic tutoring. 



THE STUDENT MODEL 

Before a session with the student begins, GUIDON 
uses MYCIN to "solve" the case to be presented to the 
student. The results of this background consultation, 
consisting of MYCIN'S rule conclusions and its 
records of why rules did not apply, are reconfigured 
into an explicit AND/OR tree of goals and rules so 
that the rules are indexed both by the goals they 
conclude about and the subgoals or data needed to 
apply them. During the tutorial session, as the student 
inquires about the patient and receives more case data, 
this same information is used to drive MYCIN'S rules 
in a forward direction. Thus, at any time, some of the 
rules MYCIN uses for determining, say, the type of the 
infection, will have led to a conclusion, while others 
will require more information about the patient before 
they can be applied. 

This record of what the expert (i.e., MYCIN) 
"knows" at any time during the student-run con- 
sultation forms the basis for evaluating a student's 
partial solutions and providing assistance. Such an 
overlay model (See Carr and Goldstein, 1977) assumes 
that the student's knowledge is a subset of MYCIN'S 
knowledge and that* there are unique reasoning steps 
for making any particular deduction. Neither 
assumption is always correct; the rule set nevertheless 
provides a first-order approximation to the student- 
modeling problem. 

The three components of the student model are 
shown in Figure 1. The three components are stored as 
properties of <*ach rule in the knowledge base. The first 
component, the cumulative record of whether a 
student knows a rule, is called the USE-HISTORY 
property of the rule. It is the program's belief that, if 
the student were given the premise of the rule, he 
would be able to correctly, in the abstract; draw the 
proper conclusion. USE-HISTORY is primed by the 
student's initial indication of his level of expertise, 
which is matched against "difficulty ratings" 
associated with each rule. Like the other two com- 
ponents, the USE-HISTORY property of a rule is 
represented as a certainty factor (the same belief 
measure used in MYCIN'S rules) that combines the 
background evidence with the implicit evidence 
stemming from needs for assistance and verbalized 
partial solutions, as well as the explicit evidence 
stemming from a direct question that tests knowledge 
of the rule. 
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Figure 1 . Maintenance relations for student-model components (Clancey, 1979b). 



The second component, called STUDENT- 
APPLIED?, records the program's belief that the 
student is able to apply the rule to the given case, that 
is, that the student would refer to this rule to support a 
conclusion about the given goal. Thus, there is a 
distinction between knowing a rule (USE-HISTORY) 
and being able to apply it, since the student may know 
which subgoals appear in the rule but be unable to 
achieve tHem. STUDENT-APPLIED? is determined 
once for each rule during a case at the time MYCIN is 
able to apply the rule. (The evidence considered is: Is it 
believed that the student knows the rule [USE- 
HISTORY]? Was the rule mentioned during this 
sesson? Has it been discussed in previous tutorials? Is 
there a subgoal that the student is not believed to be 
able to determine?) 

The third component of the student model, called 
USED?, is^ relevant whenever the student states a 
partial solution (a list of possible diagnoses, not in- 
tended to be complete). It records the program's belief 
that the student would mention a rule if asked to 
support his partial solution. This component combines 
indirect evidence by comparing conclusions made by 
rules with the student's conclusions, the record of 
what rules the student is believed to be able to tise 
(STUDENT-APPLIED?), and evidence that the 
student may have remembered to apply the rule in this 
case (e.g., the rule mentioned earlier in the dialogue). 
This combined evidence affects how the program 
responds to the partial solution and feeds back into the 
USE-HISTORY Component of the student model. 



Discourse Procedures and Alternative Dialogues 

The student is allowed to explore MYCIN'S 
reasoning by using options like FACTORS, shown 
earlier in the protocol excerpts However, the tutor is 
not a simple, passive, information-retrieval system. In 
addition to clearly laying out data andinferences, the 
tutor has to reason about what constitutes reasonable, 
expected elaboration on the basis of .what has been 



previously discussed. For GUIDON'S rule-based 
approach, this takes the form of selecting which rules 
and rule clauses to mention and deciding whether to 
ihtreducela goal for detailed discussion or just to offer 
a sunmaw of evidence. In the excerpt, for example, 
GUlI^Ntyroyi^-deu <s for an inference (rule 148) 
by offering to supporjL<achieved preconditions that 
were not mentioned in the tutorial dialogue up to that 
point. 

Similarly, when'the student takes the initiative by 
saying he has determined some subgoal, the tutor 
needs to determine what icsponse makes sense, based 
on what it knows about the student' sjcnowledge and 
shared goals for the tutorial sesson (topics or rules to 
discuss). The tutor may want to hold a detailed 
response in abeyance, simply v acknowledge the 
student's remark, or probe him for evidence that he 
does indeed know the fact in question. Selection 
among these alternative dialogues might require 
determining what the student could have inferred from 
previous interactions and the current situation. In , the 
dialogue excerpt shown above, GUIDON decides that 
there is sufficient evidence that the student knows the 
solution to a relevant subproblem so that detailed 
discussion and probing are not necessary. 

Decoupling domain expertise from the dialogue 
program, an approach used by all ICAI systems, is a 
powerful way to provide flexible dialogue interaction. 
In GUIDON, discourse procedures formalize how the 
program should behave in general terms, not in terms 
of the data or outcome of a particular case. A 
discourse procedure is a sequence of actions to be 
followed under conditions determined by the com- 
plexity of the material, the student's understanding of 
the material, and tutoring goals for the session. Each 
option available to the student generally has a 
discourse procedure associated with it. 

For example, if the student indicates, via the 
IKNOW option, that he has a hypothesis about some 
subgoal but MYCIN has not yet been able to make a 
decision, the procedure for requesting and evaluating a 
student's hypothesis is invoked. Otherwise, if MYCIN 
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has reached (he same conclusion, the procedure for 
discussing a completed topic is followed. Whether or 
not the student will be probed for details depends on 
the model that the tutor is building of the student's 
understanding (considered below). 



COMPLETEDGOAL.PROC005 
Purpose: Discuss final conclusion for a goal. 
Step 1 : Decide whether to finish with a summary. 
Step 2: Discuss final hypothesis for the goal. 
Step 3: c Wrap up discussion or record completion. 

Figure 2. Discourse procedure for completing a goal 
discussion. 

The procedure for ending discussion of a topic is 
v paraphrased in Figure 2. Conditional actions in 
discourse procedures are expressed as tutoring rules (t- 
rules). T-rules decide whether an action should be 
taken, and when this involves invoking another 
discourse procedure, other t-rules will decide what 
should be said. For example, the second step of the 
procedure COMPLETEDGOAL decides whether to 
give the student the answer or to ask him to make a 
hypothesis. Figure 3 shows the t-rule that caused 
GUIDON to acknowledge the student's statement 
about what he knew in the dialogue illustrated above, 
rather xhan ask for details. To ask about and evaluate 
the student's hypothesis, another discourse procedure 
would have been invoked. Of course, the discourse 
procedure for discussing a completed topic is invoked 
from many other procedures besides the one 
corresponding to the IKNOW option: It may be in- 
voked in the course of giving details about how a 
subgoal is determined, in responding to a student's 
/ hypothesis for a subgoal, when the program detects 
that the current subgoal (topic) is substantially 
completed (enough data have been given to make a 
strong conclusion), and so on. 



T-RULE5.02 Directly state single, known rule.- 

If 1 ) There are ryles having a bearing on this goal that 

have succeeded and have not been discussed, and 

2) The number of rules having a bearing on this goal 
that have succeeded is 1, and 

3) There is strong evidence that the student has ap- 
plied this rule 1 , 

THEN Simply state the rule and its conclusion " 

Figure 3/ T-rule for deciding, how to complete 
discussion of a topic. 

o 
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Responding to Partial Solutions 

Shown below is an annotated excerpt demonstrating 
how the program responds to partial solutions stated 
by the student. Tutoring rules are used both to update 
the model and to select the appropriate discourse 
procedures for responding. In this excerpt, while the 
student's hypothesis for a subproblem is correct, 
nevertheless, after analyzing the previous history of 
interaction with the student, GUIDON is not sure that 
the student really knows why his hypothesis is correct. 
Therefore, instead of accepting this answer, GUIDON 
requests relevant factors to support the hypothesis and 
probes further to see if the student knows why an 
alternative hypothesis is discredited. 

GUIDON asks the student to state a hypothesis for a 
subgoal. The student may have volunteered to do this, or 
the program may be testing (he student 's understanding. 

What is the type of the infection? 

Bacterial 

The program asks the student to support this hypothesis 
either because it is incorrect or, as in this case, the student 
model indicates that the student may not know all of the 
relevant rules. (GUIDON changes the prompt for student 
input to "» " to indicate that specific input is expected 
and student-initiated options are not.) 

What facts about this case tell you that the type of the in- . 

lection is bacterial? 

(Enter one factor per line.) 

»PMNs 

The percent of PMNs in the CBC is not relevant for deter- 
mining the type of the infection. 
» PMNs in trie CSF 
>>Cercbral-fluid glucose value 
» Protein of the CSF 

These factors are related to the rules used by MYCIN to 

conclude that the infection is bacterial, and the student 

model is updated accordingly. 
You're right about the CSF protein, the percent of PMNs in 
the CSF, and the CSF glucose value; however, you should 
have mentioned the fact that the meningitis is partially 
treated, the WBC from the CSF, and the fact that the 
simultaneous blood glucose value is not known. 

Other hypotheses are then discussed, 
MYCIN rejected some possibilities you did not mention: 
What facts about this case tell you that the type of the in- 
fection is not viral? ' 

Referring to the hiles that MYCIN uses to solve 
subproblems (such as determining whether a 
meningitis infection is bacterial, fungal, viral, or 
tuberculous), GUIDON decides which of these rules, if 
any, might, have been us?d by the student. That is, 
what inference chains are consistent with the student's 
behavior? This analysis is complicated by the fact that \ 
a particular hypothesis about the problem may be 
indicated by more than one rule, or negative evidence 
may outweigh positive evidence. 

A potential weakness of the GUIDON program is 
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that it attempts to explain the student's behavior solely 
in terms of MYCIN'S rules. If the student is basing his 
questions and hypotheses on incorrect rules, GUIDON 
is -not able to formulate these rules and address them 
directly. It is possible as well that the student's con- 
cepts are different from MYCIN'S, so his conclusions 
might be correct, but he will want to support them 
with reasoning that is different from MYCIN'S. This 
could involve something as simple as wanting to refer 
to the patient's age in general terms (infant, 
adolescent), while MYCIN recognizes only precise, 
numerical ages. 

Modeling medical reasoning in terms of an alter* 
native rule set (not just a subset of MYCIN'S rules) is a 
theory-formation problem that/goes beyond the 
current capabilities of AI. Itjrpossible that the ap- 
proach followed by Stevens, Collins, and Goldin 
(1982) of collecting data about student misconceptions 
and then incorporating these variations into tne 
modeling process will prove tenable for the medical 
domain. 

Opportunistic Tutoring and Pedagogical Style 

It is some lmes advantageous for the tutor to take 
the initiative to present new material to the student. 
This requires that the tutor have presentation methods 
that opportunistically adapt material to the needs of 
the dialogue. In particular, the tutor has to be sensitive 
to how a tutorial dialogue fits together, including what 
kinds of interruptions and probing are reasonable and 
expected in this kind of discourse. GUIDON 
demonstrates its sensitivity to these concerns when it 
corrects the student before quizzing him about 
"missing hypotheses," asks , him questions about 
recently mentioned data to see if he understands how 
to use them, quizzes him about rules that are related 
(by premise and action) to one that has just been 
discussed, follows up on previous hints, and comments 
on the status of a subproblem after an inference has 
been discussed ("Other factors remain to be con- 
sidered..."). - 

There are many subtle issues — when to interrupt 
the student, how much to say, and the like — that 
constitute a pedagogical style and are implicit in 
GUIDQN's teaching rules. For example, several 
tutoring rules in different situations may present short 
orientation lectures, but nowhere does GUIDON 
reason that its interaction will be of the tutorial type, 
which provides orientation when appropriate, in 
contrast with the coaching type {e.g., Burton and 
Brown, 1979), which only makes interruptions. For 
this reason, it is useful to summarize the set of tutoring 
principles that appear implicitly in the tutoring rules: 
1 . Be perspicuous: Have an economical presen- 
tation strategy, provide lucid transitions, and 



adhere to conventional discourse patterns. 

2. Provide orientation to new tasks by top-down 
refinement: Provide' the student with an 
organized framework of considerations he 
should be. making, without giving away the 
solution to the problem (important factors, 
subgoals, size of the task), thus challenging the 

/student to examine his understanding con- 
structively. 

3. Strictly guide the dialogue: Say when topics are 
* finished and inferences are completed, as op- 
posed to letting the student discover transitions 
for himself. 

4. Account for incorrect behavior in terms of 
missing expertise (as opposed to assuming 
alternative methods and strategies): Explain 
clearly what is improper from the tutor's point of 
view (e.g., improper requests for case data). This 
is, of course, more a statement of how GUIDON 
models the student than a principle of good 
teaching. 

5. Probe the student's understanding when you are 
not sure what he knows, when you are 
responding to partial student solutions: 
Otherwise, directly confirm or correct the . 
solution. 

6. Provide assistance by methodically introducing 
small steps that will contribute to the problem's 
solution: 

a\ Assistance should at first be general, to remind 
the student of solution methods and strategies he 
already knows; 

b. Assistance should encourage the student to 
advance the solution by using case data he has 
already been given. 

7. Examine the student's understanding and in- 
troduce new information whenever there is an 
opportunity to do so. 

Case and Domain Independence 

Patient cases are entered into the MYCIN system for 
receiving a consultation or for testing the program, so 
the case library is available to GUIDON at ho cost. 
This provides over 100 patients that GUIDON can 
distuss, clearly demonstrating the advantage that 
ICAI has over the traditional computer-based- 
instruction approach in which each lesson .must be 
designed individually. 

Besides being able to use the teaching procedures to 
tutor different cases, GUIDON can provide tutorials 
in any problem area for which a MYCIN-like 
knowledge base of decision rules and fact tables has 
been formalized (see van Melle, 1980). This affords an 
important perspective on the generality of the 



11 



WILLIAM J. CLANCEY 



discourse and pedagogical rules. 

Experimental tutorials using knowledge bases in two 
other domains — structural analysis (SACON) and 
pulmonary function diagnosis (PUFF) — have 
revealed that the effectiveness of discourse strategies 
for carrying on a dialogue economically is determined 
in part by the depth and breadth of the reasoning tree 
for Solving problems, a characteristic of the rule set for 
each domain. When a solution involves many rules at a 
given level, for example, when there are many rules v.o 
determine the organism causing the infection, the tutor 
and student will not have time to discuss each rule in 
the same degree of detail. Similarly, when inference 
chains are long, an effective discourse strategy will 
entail summarizing evidence on a high level, rather 
than considering each subgoal in the chain. 



RESULTS 

GUIDON demonstrated that teaching knowledge 
could be treated analogously to the domain expertise 
of consultation systems: It can be codified in rules and 
built incrementally by testing it on different cases. The 
framework of tutoring rules organized into discourse 
procedures worked well, indicating that it is suitable to 
think of a tutorial dialogue as being separated into 
relatively independent sequences of interaction. 
Moreover, the judgmental knowledge for constructing 
a student model can also be captured in rules utilizing 
certainty factors, showing that the task of modeling a 
student bears some relation to MYCIN'S task of 
diagnosing a disease. 

In contrast to GUIDON'S teaching knowledge, the 
evaluation of MYCIN'S rule set for this application 
was not so positive. While MYCIN'S representational 
metaknowledge made possible a wide variety of 
tutorial activity, students find that the rules are dif- 
ficult to understand, remember, and incorporate into a 
problem-solving approach. These difficulties 
prompted an extensive study of MYCIN'S rules to 
determine why the teaching points were not as clear as 
had been expected. GUIDON researchers discovered 
that important structural knowledge (hierarchies of 
data and diagnostic Hypotheses) and strategic 
knowledge (searching the -roblem space by top-down 
refinement) were implicit in the rules. That is, the 
choice and ordering of rule-premise clauses constitute 
procedural knowledge that brings about good 
problem-solving performance in a MYCIN con- 
sultation but is unavailable for teaching purposes. 
Rather than teaching a student problem-solving steps 
(rule clauses) by rote, it is advantageous to convey an 
approach or strategy for bringing those steps to mind 
— the plan that knowledge-base authors were 



following when they designed MYCIN'S rule set. To 
make this implicit design knowledge explicit, a new 
system, NEOMYCIN'(Clan<;ey and Letsinger, 1981), 
is being developed that separates out diagnostic 
strategy from domain knowledge and makes good use 
of hierarchical organization of data and hypotheses. 

Moreover, besides reconfiguring MYCIN'S rules so 
thai knowledge is separated out and represented more 
declaratively, it is necessary to add knowledge about 
the justification of rules. Justifications are important 
as mnemonics for the heuristic associations, as well as 
for providing an understanding that allows the 
problem solver to violate the rules in unusual 
situations. 

finally, NEOMYCIN has additional knowledge 
a-, out disease processes that allows it to use the 
strategy of "group and differentiate" for initial 
problem formulation, in which the problem solver 
must think about broad categories of disorders and 
consider competing hypotheses . that explain ,the 
problem data. Thus, we want to teach the student the 
knowledge a 1 , human would need to focus on in- 
fectious-disease problems in the first place, essentially 
the knowledge v (pr?viously unformalized) that a human 
needs to ine MVCIN appropriately. 

In conclusion, GUIDON research sets out to 
demonstrate the advantages of separate, explicit 
representations of both teaching knowledge and 
subject material. The problems of recognizing student 
Misconceptions aside, this research demonstrated that* 
simply representing in an ideal way what to teach the 
student is not a trivial, solved problem. An un- 
structured set of production > rules is inadequate. 
GUIDON's teaching rules are organized into 
procedures; NEOMYCIN'S diagnostic rules are 
hierarchically grouped by both premise and action and 
are controlled by meta-rules. GUIDON research 
demonstr£te€khat the needs of tutoring can serve, as a 
"forcing functibn" to direct research toward more 
psychologically valid representations of domain 
knowledge, which potentially will benefit those aspects 
of expert-systems research that require human in- 
teraction, particularly explanation and knowledge * 
acquisition. 



GUIDON is described full> by Clancey (1979b): a shorter discussion is 
given in Clancey ' (1979a). Clancey and Letsinger (1981) describe the 
NEOMYCIN research. The study of MYCIN'S rule base leading upto this 
new system and some methodological consideiations are provided by Clancey 
(1983. in press a). 
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1 Dr. 0**14 Kleras 
Ocpartaent of Pjyc.piajy 
University of Arisen* 
Twcw, AX 8S721 

1 Or, Valtar JClatsch 
Dopartaant of Psychology 
Onlvsrslcy of Colorado 
Boulder, CO 80302 

1 Dr. Stephen Ioeslya~> 

iX3i ULUla JaMi lUl 

33 Urbiand St. s 
Cambridge, !U 92131 

I Or. r»c Usflty 
The Bnhotlce laetltote 
C«nt; lts-1 illoo Unlverelty 
Pltt.burjh. PA 13213 

1 Dr. Jill Lorkln 
SiHrtMac of Feycbology 
Cermet lo Halloo Oalverelty 
ricttWrsh, FA 13213 

1 Or. Alaa Lesgold 
loaralag BAD Cantor 
Onlvarelty of Pittsburgh 
3939 O'Bara Stroot 
Plttebur*h, Fa, 13260 

1 Dr. Jim Vevln 
Oalverelcy of California 

•t Saa 01ogo 
Labotatory fof Comparative 

lasnan Co gale Ion - 500 3A 
U Jella. CA 92093 

1 Br. Michael Lovlae 
Damartmoac of Educational Psychology 
210 tdacatlon Bldg. 
Onlverelty of IlUoolo 
Caaamalga, XL 61801 

1 Dr. Morda C. Limn 
Lavrenca laU of Sclaoca 
Onlverelty of Callforala 
Berkeley. CA 94720 

1 Dr. too Lyon 
ATOL/OT (TOX) 
VUUoaa An. AZ 33223 

1 Br. Joy MeClollaad 
Department of Psychology 
KIT 

Ceabrldge. HA 02139* 

1 Or. Jama* a. miller 
Soaootor*Tb»ttfbt Corporation 
1721 Voat Plaaa Bighvay 
Plane. TX 79073 

i Or. Hark Millar 
Coaeotjlr+Tbeught Corporation 
1721 Vaot rlaao Parkway 
rlaao,. XX 79073 

1 Or. Toa aorta 
laroa PABC 

3333 Coyati IU1 load 
Falo Alto, a 94306 

1 Or. Uloa iiaro 
Behavioral Technology Laborat^Hss 
1043 Elena Ave. , Fourth Floor 
tadoodo Beech. CA 90277 

1 Or. Ooaald A Soman 
Cognitive Sctence, C-013 
Doiv. af Callforala, Saa Olofo 

, La Jella. CA 92093 



1 Dr. Jeese Orleas'cy 

Institute for Defense Anelyees 
1801 N. Beeuregerd St. 
Alexandria, VA 22911 

1 Prof* Seyaour Peperc 
20C-139 

Kaseachusette Institute of Technology 
Ceabrldge, HA 02199 

1 Or* Nancy Pennington 
— Onlvorstty ^f Calcogo 

Graduate School of Business 

1101 t. 38th St. 

Chicago, XL 60697 

l ot. rem pou oh 

DEPT. OF PSYCHOLOGY 
WIVEtSlTY 0P COtbftADO 
3901DEE, CO 80909 

1 Or, Prod Bc&f 
Pbyslco Ooparsaoot 
Onlverelty of California 
Berkeley, CA 94720 

1 Or, Lauren Re snick 

use 

Onlverelty of Pittsburgh 
9939 O'Eara Strtot 
PWaborgh, PA 1321 m 

1 Or. Jalf Richardson : 
Denver Baaoarcb Institute 
Oalvsrslty of fervor 
Denver, 00 80206 

1 Mary S. Ill ay ' 
Prograa Is Cognitive Science 
Cantor for auaaa Information *?j:tsiiag 
Onlverelty of Callfornio, Saa Otogo 
U Jolla, CA 92093 

1 Or. Aairov toes 
Aaarleaa Institutes for aat torch 
1033 Taovnaa Jeffereoo St. BV 
tfaohlafcaa, 0C 20007 

1 Or. Eraac t. tothkopf 
toll Laaoratorlee 
ferrey Bill, ttJ 07974 

1 Or. Vllllaa 1. toaoo 
Georgia Institute of T:rKaology 
School of ladustrlal 4 Systems 

lag lnenrlng 
Atlaaca, CA 30332 

1 Or. Oav'_-j toaelharc * 
Cantor lor Haaia Information Processing 
Onl-r. -of Callformle, 5m -Kej» 
Le Joi'ie, CA 92093 

I Or, Kc&enl J, Saaat 
Perceptr amies, lac 
6271 Varlal Aveuoo 
0™<Hnrt Hills, *U 91364 

1 Dr^-togax Schank 
Y*la Cclvarolsy 
Department of Coapotor Scloaco 
P.O. Scot 2136 
Vow Bavaa, CT 06320 

1 Or. Valtor Schneider 
Psychology Department 
603 E. Daniel 
Cheasalga. XL 41820 ~ 

1 Or. Alon Scboeofsld 
Katboaatleo aai Uucatloo ■< 
Tha Onlvorolty of Rocaootor 
loebsstor, Sff 14627 



I Hr. Colla Shopptrd 
Applied Psychology Dolt 
Adalralty Marios Tochaology Est. 
Toddlngsoa, Sliddloocx 
DoitsS tingdos 

1 Or. I. Vallaco SlaUko 
Prograa Dlroetor 

HAopowor laaosrch and Advisory Sortlcoo 

Soitbsoalan Institution 

801 Jibrth Pitt Stroot 

Alexandria, VA 22914 *~ 

1 Or. tdvard I. ' Soith 
Bolt Boraaak 6 5ovaaa, Int. 
30 Moultoo Strost 
Caabrldgo, HA 0213S 

1 Or. licasrd Snov 
School of Uueotlon ■ ° 

Stanford Oalvorslty 
Stanford, CA 94303 

I Or. Sllott Solovsy 
Talo UoWorolty 
DapartaaoC of Computer Science 
P.O. Box 2138 
Kev lav in. CT 06923 

r 

1 Or. Katbryn T. Spoobr 
Psycbology Dopertaeot 
Brown OoWorelty 
Providence, U 02912 

1 Or. Robert Sternberg 
Dept. of Psycbology » 
Tale Oniveraity 

Box UA, Yale Stetlon 1 
2lev ttisan, CT 05320 

1 Dr. Albert Stevens 
Bolt 3ar*.iVt 4 Sewaaa. Inc. 
19 .Sou Iron Sc. 
Cambridge, HA 01238 

1 Oayld B. Stone, Ph,0. 
lasaltloe Corporacloa 
7669 Old Sprlnghouse toad 
HcLeaa, VA 22102 

1 Or. Cltoal.Teteuofca 

Conpstor la ted E£u:etloa laaaercb Lab 
292 Eaf laoerlng ftoeearcb laboratory 
Orbaaa, XL 61991 

1 Or. Parry U. Tboradyaa 
Percopcronlcs, Inc.. 
343 mddlefleld Uoaa, Sol to 140 
Henlo Park, CA 94025 

1 Or. Domglas Tevoo 
Onlv. of So. California 
Behavioral Technology Labo 
2843 S. Elaaa Ave. 
Hadondo Beach, CA 90277 

I Or u . Cart Von Uha 
Xerox PAAC 

3333 Coyote till toad 
Palo Uta, CA 94304 

I Or, Kalch T. tfescoart 
Perce ptronlcs, lac. 
343 rtlddlef it Id Road. Sulca 140 
Henlo Para, CA 94023 

1 Vllllaa B. Whitten 
Bell Laborotorlee 
20-610 

loladol, Rl 07733 
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I Or. Iboui Vidua* 
OtaartMat of Hycbolory 

WntUty «f C*llforul« 
*03 lllgard* Avaaaa 
Ua AajtUs, CA 90024 

I Or. Xlk* Vllllaa* 
telt ft«r«n«k ft lUuan 
10 Boolean Sc. 
Co»rUs*» IU J4304 

t Or. Jo««»h Ileal 
AIjh*t«ch, la:. 
2 torllaftoa Ixtcuclv* C«ac«r 
111 Hlddla««t Turaalk* 
terllaftaa, HA 01903 
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